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ABSTRACT 

Vir 

The  solvability  of  a  linear  program  is  characterized  in  terms  of  the 
existence  of  a  fixed  projection  on  the  feasible  region,  of  all  sufficiently 
large  positive  multiples  of  the  gradient  of  the  objective  function.  This 
projection  turns  out  to  be  the  normal  solution  obtained  by  projecting  the 
origin  on  the  optimal  solution  set.  By  seeking  the  solution  with  least 
2-norm  which  minimizes  the  1-norm  infeasibility  measure  of  a  system  of 
linear  inequalities  or  of  the  optimality  conditions  of  a  linear  program,  one 
is  led  to  a  simple  minimization  problem  of  a  convex  quadratic  function  on 
the  nonnegative  orthant  which  is  guaranteed  to  be  solvable  by  a  successive 
overrelaxation  (SOR)  method.  This  normal  solution  is  an  exact  solution  if 
the  original  system  is  solvable,  otherwise  it  is  an  error-minimizing  solu¬ 
tion.  New  computational  results  are  given  to  indicate  that  SOR  methods  can 
solve  very  large  sparse  linear  programs  that  cannot  be  handled  by  an  ordinary 
linear  programming  package.  _ _ _ _ - 

AMS  (MOS)  Subject  Classifications:  90C05,  90C06,  90C20,  15A39 

Key  Words:  Linear  programming,  linear  inequalities,  large  scale  systems, 
quadratic  programming 
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SIGNIFICANCE  AND  EXPLANATION 


Linear  programming  problems  may  not  have  exact  solutions  due  to 
inaccuracies  in  the  data.  By  considering  approximate  solutions  which 
are  closest  to  the  origin  we  have  come  up  with  a  simple  but  powerful 
iterative  method  that  can  solve  exactly  (if  the  linear  program  is 
solvable)  or  approximately  (if  the  linear  program  is  unsolvable)  very 
large  sparse  linear  programs  that  cannot  be  solved  by  ordinary  linear 
programming  techniques  such  as  the  simplex  method. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


NORMAL  SOLUTIONS  OF  LINEAR  PR06RAMS 
0.  L.  Manga sari  an 


1.  Introduction 

A  normal  solution  to  a  linear  program  Is  an  exact  solution  with  some 
least  norm  property  If  the  linear  program  Is  solvable,  otherwise  It  Is  an 
approximate  solution  with  some  least  norm  property  also.  By  an  approximate 
solution  we  mean  a  point  which  minimizes  a  measure  of  satisfaction  of  the 
optimality  conditions  of  the  linear  program.  By  considering  normal  solu¬ 
tions  we  are  led  to: 

(1)  Iterative  successive  overtaxation  (SOR)  methods  capable  of 
solving  very  large  linear  programs. 

(11)  Approximate  solutions  to  poorly  posed  or  uns olvable  linear 
programs. 

(Ill)  A  stable  solution  or  approximate  solution,  to  a  linear  program, 
endowed  with  a  least  norm  property. 

For  solvable  linear  programs  our  normal  solution  Is  essentially  equiv¬ 
alent  to  that  of  Tikhonov  and  Arsenin  [16]  which  they  obtain  by  solving  an 
asymptotic  problem  [16,  Theorem  1,  p.  226],  whereas  our  solution  Is  obtained  by 
solving  a  simpler  exact  problem,  problem  (2.2)  for  any  ee(0,e]  for  some  i>0 
(Theorem  2.1).  Tikhonov  and  Arsenin's  weaker  asymptotic  result  comes  about 
because  they  square  the  objective  function  of  the  linear  program  In  their 
regularization  problem  [16,  p.  226]  and  thereby  lose  an  essential  exact 
feature  of  our  problem  (2.2).  Tikhonov  and  Arsenin  also  do  not  consider  the 
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Important  case  of  possibly  unsolvable  linear  programs  (Section  4),  nor 
do  they  give  explicit  computational  methods  for  solving  their  asymptotic 
problem. 

We  outline  now  our  principal  results  and  their  relation  to  other  work. 
In  Section  2  we  consider  normal  solutions  of  solvable  linear  programs.  In 
Theorem  2.1  we  give  a  complete  characterization  of  the  solvability  of  a 
linear  program  In  terms  of  a  2-norm  projection  on  the  feasible  region  of  a 
sufficiently  large  but  finite  positive  multiple  of  the  gradient  of  the 
objective  function.  This  projection  turns  out  to  be  fixed  and  equal  to  the 
unique  2-norm  projection  of  the  origin  on  the  optimal  solution  set.  Part 
of  Theorem  2.1,  a(i),  follows  readily  from  [13,  Theorem  1],  while  Its 
converse,  part  a(il),  which  is  essential  for  a  comprehensive  justification 
of  the  linear  programming  SOR  Algorithm  2.3,  has  not  been  available  before. 
Theorem  2.2,  which  follows  from  Theorem  2.1  and  quadratic  programming 
duality,  characterizes  the  solvability  of  a  linear  program  In  terms  of  the 
solvability  of  a  convex  quadratic  function  minimization  on  the  nonnegative 
orthant  (2.7)  without  any  a  priori  assumptions  regarding  the  solvability 
of  the  linear  program  (2.1)  as  was  the  case  in  [11,12].  In  addition. 
Theorem  2.2  gives  the  complete  basis  for  the  linear  programming  SOR 
Algorithm  2.3  and  its  convergence  (Theorem  2.4)  thereby  sharpening  earlier 
convergence  results  [11,12]. 

In  Section  3  we  turn  our  attention  to  a  system  of  possibly  inconsis¬ 
tent  linear  Inequalities  (3.1)  and  reduce  It  to  the  problem  of  finding 
the  unique  least  2-norm  solution  of  the  problem  of  minimizing  the  1-norm 
Infeasibility  measure  of  (3.1).  The  principal  advantage  of  this  approach 
Is  that  It  leads  to  the  SOR  Algorithm  3.1  which,  unlike  most  other 
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iterative  procedures  [1 .15,33  which  require  an  a  priori  consistency  as¬ 
sumption.  will  converge  no  matter  whether  the  original  system  (3.1)  Is 
consistent  or  not.  In  either  case  Algorithm  3.1  will  give  an  exact  or 
approximate  solution  with  least  2-norm  (Theorem  3.2).  Among  the  poten¬ 
tial  useful  applications  of  this  approach  Is  In  the  Image  reconstruction 
techniques  of  tomography  [7,8]  which  require  the  solution  of  enormous 
sparse  systems  of  linear  equations  with  nonnegative  variables.  Most 
current  Iterative  techniques  for  the  tomography  problem  [7,8,2]  need  an 
a  priori  assumption  regarding  the  consistency  of  the  original  system. 

In  contrast  our  Algorithm  3.1  needs  no  such  assumption. 

In  Section  4  we  consider  possibly  unsolvable  linear  programs  and 
reduce  their  solution  to  finding  the  least  2-norm  primal-dual  solution 
which  minimizes  the  1-norm  of  the  optimality  conditions  of  the  given 
linear  program.  This  approach  leads  to  an  SOR  algorithm  that  Is  guaran¬ 
teed  to  work  whether  the  original  linear  program  Is  solvable  or  not.  In 
either  case  It  will  give  an  exact  or  approximate  solution  with  least 
2-norm. 

Finally  in  Section  5  we  give  some  numerical  comparisons  for  one 
version  of  our  linear  programming  SOR  algorithm  with  the  XMP  version  [14] 
of  the  revised  simplex  method  for  medium  and  large  size  sparse  linear 
programs.  These  comparisons  Indicate  that  SOR  methods  can  solve  very 
large  sparse  linear  programs  that  cannot  be  solved  by  an  ordinary  linear 
programming  package. 

We  briefly  describe  the  notation  used.  All  matrices  and  vectors  are 
real.  For  the  m*n  matrix  A  we  denote  row  1  by  Aj ,  column  j  by  A.j 
and  the  element  In  row  1  and  column  j  by  A^j.  For  x  in  the  real 
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n-dlmenslonal  Euclidean  space  Rn,  x^  denotes  element  1  for  1*1 

and  x+  denotes  the  vector  with  components  (x+)^  •  max{x^,0>,  1*1* . n. 

Vectors  are  either  row  or  column  vectors  depending  on  the  context.  For  x 

n  n 

and  y  In  R  ,  xy  denotes  the  scalar  product  £  x^,  while  ||x||t 

n  I  1,1 

for  1  <  t  <  «  denotes  the  t-norm  (  l  |x,  (*)*  ,  ||x||  denotes  an  arbl- 

"  ”  1-1  1 

trary  but  fixed  norm  on  Rn  and  }{ A||  denotes  the  subordinate  matrix  norm 

max  II Ax ||  for  an  mxn  matrix.  The  vector  e  will  denote  a  vector  of 

11*11-1 


ones  In  any  real  Euclidean  space.  r”  will  denote  the  nonnegative  orthant 
{x|x«Rn,  x>0}.  For  a  point  c  in  Rn*  a  closed  set  X  In  Rn  and  a 
number  t  e  [1  ,<»]  the  t-norm  projection  p^(ctX}  of  the  point  c  on  X 
Is  defined  by 


||c-pt(c,X)||t  -  min  ||c-x||t 
Xc  X 


For  a  function  f:  Rn  -*•  R  which  Is  twice  differentiable  on  Rn,  Vf(x) 

denotes  the  n-dlmenslonal  gradient  vector  at  x  with  components 

7  f(x),  1-1,.... ,n,  and  7  f(x)  denotes  the  nxn  Hessian  matrix  at 
1  , 

x  with  elements  (7£f(x))ij,  i,j»l, _ ,n. 


S 


» 
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2.  Normal  solutions  of  solvable  linear  programs 
We  consider  here  the  linear  program 

(2.1)  maximize  cx  subject  to  x«X:»  {x|xeRn,  Ax<b,  x>0} 

where  b  and  c  are  given  vectors  In  Rm  and  Rn  respectively  and  A 
Is  a  given  mxn  real  matrix.  Let  X  denote  the  (possibly  empty)  optimal 
solution  set  of  (2.1).  We  shall  assume  throughout  this  section  that  this 
linear  program  Is  feasible,  that  Is  X  Is  nonempty.  We  begin  with  the 
following  fundamental  and  geometrically  plausible  result. 


2.1  Theorem  Let  the  linear  program  (2.1)  be  feasible.  Then 

a.  (1)  max  cx  has  a  solution  >  0:  p,(£ .X)  *  p9(0,X)  for  all  ee(0,e] 

x«X  1  z  z 

(11)  max  cx  has  a  solution)*"  3e  >  0,  x:  p,(-,  X)  ■  x  for  all  ee  (0,e] 
x«X  /  z  e 

and  x  »  p2 ( 0 ,X )  / 

where  P2(x,X)  denotes  the  2-norm  projection  of  x  on  X. 

b.  sup  cx  ■  ®  j|p2(§,X)||  -*>  »  as  e  0+ 

xeX  c  e 

Proof 

a ( 1 ) :  By  noting  that  pgfpX)  Is  a  solution  of  either  of  the  equivalent  problems 

(2.2)  min  ||x-jf-||2  «-*  min  -  cx  +  f  |{ x|| 2 

XeX  e  2  x«X  2  2 

the  Implication  of  a(1)  follows  from  Theorem  1  of  [13], 


a(  11 ) :  Since  x  ■  pg^.X)  for  all  ee(0,e],  then  there  exists 
(u(e),  v(e))cRm+n  such  that  (x,  u(e),  v(e))  satisfies  the  Karush-Kuhn- 
Tucker  conditions  [9]  for  (2.2),  that  Is 
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(2.3)  ex  -  c  +  A^u(e)  -  v(e)  ■  0,  v(e)x  ■  0,  Ax  <  b,  u(e)(Ax-b)  ■  0, 

(x,  u(e),  v (e) )  >  0,  Ve  e(0,e] 

By  the  fundamental  theorem  for  the  existence  of  basic  feasible  solutions 
for  linear  equations  with  nonnegative  variables  [6,  Theorem  2.11],  and  the 
complementarity  conditions  u(e)(Ax-b)  *  0,  v(e)x  *  0,  it  follows  that 
there  exist  (u(e),  v(e))  satisfying  (2.3)  such  that  all  elements  of 
(u(e),  v(e))  not  corresponding  to  some  subset  of  k  linearly  Independent 
columns  of  [AT  -I]  are  zero.  Since  the  rank  of  [AT  -I]  is  n.  It 
follows  that  we  can  take  k  ■  n  and  denote  by  B(e)  as  this  "basis" 
matrix  of  n  linearly  Independent  columns  of  [AT  -I].  Hence  It  follows 
for  such  a  "basic"  solution  (u(e),  v(e))  satisfying  (2.3)  that 

||u(e)  v(e) ||  <  (||c||+i|jx||)||B(er1||  Vec(0,£] 

Since  [AT  -I]  contains  a  finite  number  of  basis  matrices  It  follows  that 
for  some  basis  matrix  B,  ||B(e)”^|J  <  ||B”^||  for  all  ee(0,i]  and 
consequently 

(2.4)  ||u(e)  v(e)||  <  (||c||  +  e|lx||  JHB'1 1|  Vcc(0,i] 

Now  let  {e1}  be  a  sequence  of  positive  numbers  in  (0,e]  converging  to 
0.  Then  there  exists  a  sequence  {(u(e^),  v(e^))}  satisfying  (2.3)  and 

(2.4)  and  hence  it  is  bounded  and  has  an  accumulation  point  (u , v ) 
satisfying 

(2.5)  -c  +  A^u  -  v  *  0,  vx  *  0,  Ax  <  b,  u(Ax-b)  *  0,  (x,u,v)  >  0 

These  are  the  Karush-Kuhn-Tucker  conditions  for  the  linear  program  (2.1) 
and  hence  x  solves  (2.1).  Since  (x,  u(e),  v(e))  also  satisfies  (2.3) 
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which  are  also  the  Karush-Kuhn-Tucker  conditions  for 
1  2 

min  jllxUg  subject  to  Ax  <  b,  x  >  0,  cx  >  cx 
with  optimal  x  ■  x  and  optimal  multiplier  vector  of  (u4e^.  v  ^  t  ~).  It 

E  E  C 

follows  that  x  »  p2(0,X). 

(b)  («-):  If  not  then  the  linear  program  (2.1)  has  a  solution  and  by  part 
a(1)  of  this  theorem  3i  >  0:  p2(| ,X)  ■  p2(0,X)  for  all  ee  (0,e].  This 
however  contradicts  the  hypothesis  that  HPgCftX)!!  '*’ *  as  e  -►  0+ . 

(■*):  If  not  then,  for  a  sequence  of  positive  numbers  (e1)  converging  to 

zero,  the  sequence  {||p2(’^'»x)|!}  Is  bounded.  By  defining  x(e*):«  p2 (-^p»X) 

we  get  that  xte1)  and  some  (u(e1),  v(e1))£ff+n  satisfy  the  Karush- 

2 

Kuhn-Tucker  conditions  for  min  I) x -  -t-  |1  for  1«1,2 . .  that  Is 

xcX  e»  2 

(2.6)  e*x(e*)-c+ATu(e* )  -  v(ei )  -  0,  v(e1)  x(e1 )  •  0,  Axfe1 )  <  b,  u(e1  )(Ax(ei)-b)  «  0 

(xte1),  uU1),  vU1))  >  0 

By  the  same  argument  as  In  the  proof  of  part  a (11 )  of  this  theorem  we  can  show 

that  the  sequence  {(uU1),  v(e*))>  satisfying  (2.6)  can  be  taken  as  bounded 

since  (x(ci))  Is  also  bounded.  Thus  the  sequence  {(xU1),  u(e*),  v(e*))} 

1$  bounded  and  has  an  accumulation  point  (x,u,v)  satisfying  (2.5).  Hence  x 

solves  (2.1)  which  contradicts  the  hypothesis  that  sup  cx  ■  ».  □ 

xcX 

By  noting  that  the  quadratic  programming  dual  [9]  to  (2.2)  Is 

(2.7)  minimize  4  ||ATu  -  v  -  c||*  +  ebu 

u.v)«Rfn  2 
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where  the  primal  and  dual  variables  x  and  (u,v)  are  related  by 

(2.8)  x  -  i-(-ATu+v  +  c)  , 

the  following  theorem  Is  a  direct  consequence  of  Theorem  2.1. 

2.2  Theorem  The  linear  program  (2.1)  Is  solvable  If  and  only  if  there 
exists  an  e  >  0  such  that  for  each  ce  (0,e]  the  quadratic  program 
(2.7)  has  a  solution  (u(e),  v (e ) )  and  such  that  the  vector  x  defined  by 

(2.9)  x:«  ^(-A^u(e)  +  v(e)  +  c)  ee  (0,e] 

Is  Independent  of  e.  In  which  case  x  -  p2(0,X). 

If  we  define  the  objective  function  of  (2.7)  by 

(2.10)  f(z):*  ^||ATu- v- clj^  ♦  ebu,  z:* 

then  we  can  prescribe  an  SOR  procedure  for  solving  (2.7)  which  in  view  of 
Theorem  2.2  solves  the  linear  program  (2.1).  The  SOR  procedure  Is  essen¬ 
tially  a  gradient  projection  algorithm  of  the  following  type 

(2.11)  zj  1  *  (zj-w(V2f(z1))jjv2^f(z]+1, . ‘zj-l’zj’ . zm+n^+* 

0  <  (0  <  2,  J«1 . . ,m+n. 

More  specifically  [12]  the  following  SOR  algorithm  for  solving  the  linear 
program  (2.1)  follows  directly  from  (2.11)  and  (2.10). 

2.3  LPSOR(A.b.c)  Algorithm  Choose  (u®,  v®)£Rm+n,  we  (0,2)  and 
e  >  0.  Having  (u*.  v1)  determine  (u1+1,  v^+1)  as  follows: 
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!*'  ■  T  <*T>.  “i*1*  "  (*T>.  u!-v’-c)*cb)j  if  A,f 

J  J  Ml  J  1-1  *•  1  i«j  4  1  J  +  J 

J  2  for  j>l 


“J*1  *  0  "  »j  *  « 


v1+1  *  (v^-aVW+c)) 


j-l , . ,m 


Note  that  Algorithm  2.3  Is  sparsity-preserving  for  it  works  with  the 
rows  of  A  only  and  the  product  AA^  need  not  be  computed. 

The  following  convergence  theorem  which  follows  from  Theorem  2.2  above 
and  [12,  Theorem  2]  sharpens  previous  LPSOR  convergence  theorems 
[11,  Theorem  3.2]  and  [12,  Theorem  4], 


2.4  LPSOR (A.  b.  c)  Convergence  Theorem 

(a)  The  linear  program  (2.1)  has  a  solution  if  and  only  if  there  exists  a 
real  positive  number  i  such  that  for  each  ee  (0,e],  each  accumula¬ 
tion  point  (u(e),  v(e))  of  the  sequence  {(u^.v1)}  generated  by  the 
LPS0R(A,  b,  c)  Algorithm  2.3  solves  (2.7)  and  the  corresponding  x 
determined  by  (2.9)  Is  Independent  of  e.  In  which  case  x  *  p2(0,X). 

(b)  If  the  linear  program  (2.1)  has  a  solution  and  Its  constraints  satisfy 
the  Slater  constraint  qualification,  that  is  Ax  <  b  for  some  x  >  0, 
then  the  sequence  {(u\v^)}  of  the  LPS0R(A,  b,  c)  Algorithm  2.3  is 
bounded  and  has  an  accumulation  for  each  ee  (0,e]  for  some  i  >  0. 


Computational  results  for  the  Algorithm  2.3  are  given  In  Section  5. 
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3.  Normal  solutions  of  possibly  Inconsistent  linear  Inequalities 
We  consider  in  this  section  the  possibly  inconsistent  system  of 
linear  inequalities 

(3.1)  Ax  <  b,  x  >  0 

where  A  Is  a  given  mxn  matrix  and  b  Is  a  given  vector  in  Rm.  If 
we  try  to  "solve"  the  above  system  by  an  SOR  [10]  procedure  applied  to  the 
obvious  2-norm  minimization  problem 

(3.2)  min  ||(Ax-b)+||*  «  min  J}Ax+y-b|l?  min  9(x,y) 

x>0  2  (x,y)>0  2  (x,y)>0 

one  needs  the  condition 

>0,  for  some  (x,y)  e  Rn+m 

to  guarantee  boundedness  of  the  SOR  iterates  [10,  Theorem  2.2],  which  by 
the  Gordan  Theorem  [9,  Theorem  2.4.5]  is  equivalent  to  the  condition  that 

(3.4)  Ax  <  0,  Of  x  >  0,  has  no  solution 

Unfortunately  this  condition  is  not  satisfied  in  general,  as  is  the  case 
when  the  feasible  region  is  nonempty  and  unbounded.  To  avoid  this 
difficulty  we  use  the  SOR  procedure  of  Section  2  to  find  the  2-norm 
projection  of  the  origin  in  Rn+in  on  the  nonempty  solution  set  of  the 
linear  program 

(3.5)  minMAm  {eyjAx-y<b} 

(x.yMl™ 


(3.3)  V6(x,y)  * 


AT(Ax  +y  -  b) 
[_Ax  +  y  -  b  _ 
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which  Is  the  equivalent  of  the  problem  of  minimizing  the  1-norm  feasibility 
of  (3.1) 


(3.6)  min  ||  (Ax -b)  || 

xeRj  1 

The  key  feature  of  this  approach  Is  that  the  SOR  procedure  will  work  no 
matter  whether  the  system  (3.1)  Is  consistent  or  not.  In  either  case  the 
SOR  procedure  will  obtain  the  unique  solution  (x»y)  of  (3.5)  with  least 
2-norm.  In  terms  of  the  original  Inequalities  (3.1),  x  Is  the  unique 
solution  of  (3.6)  which  minimizes  ||x,  (Ax  -  b)+ 1| Needless  to  say.  If 
(3.1)  Is  consistent  then  x  Is  the  unique  2-norm  projection  of  the  origin 
In  Rn  on  the  nonempty  feasible  region  determined  by  (3.1).  To  obtain  an 
SOR  procedure  for  solving  (3.5)  we  take  the  dual  of  the  quadratic 
perturbation  of  (3.5) 


(3.7) 


m1nn+«{ey+f  ll*»  yIU  Ax-y<b> 


which  turns  out  to  be  [9] 

(3.8)  min  1  ||ATu-v||*  +  i  ||u+w-e||*  +ebu»: 

(u,v.w)tRfn+* 

with  (x,y)  related  to  (u.v.w)  by 

(3.9)  x«^(-ATu  +  v),  y»l(u+w-e) 


min 

(u.v.w)eR 


m+n-Hn 

+ 


\|<(u,v  ,w) 


Since 


(3.10)  7^(u,v,w)  ■ 

A(A^u-v)  +  u  +  w-  e  +  eb 

,  ^(u.v.w)  • 

AAT+I 

-A 

I 

-(ATu-v) 

-at 

I 

0 

u  +  w  -  e 

I 

0 

I_ 

i 


It  follows  that  7i|»(0,e,Xe)  >  0  for  sufficiently  large  X  and  consequently 
the  Iterates  of  the  SOR  algorithm  of  [10,  Algorithm  3.2,  Remark  3.2]  applied 
to  3.8  will  have  an  accumulation  for  all  positive  values  of  e.  In  particular 
we  have  the  following  algorithm  and  convergence  theorem. 

3.1  LISOR(A.b)  Algorithm  Choose  (u°, v°,w°)  £  rJ4"41",  we  (0,2)  and  e  >  0. 
Having  (u^.v^.w1)  determine  (u*+1,  v*+1 ,  wi+1 )  as  follows: 

ul*1  -  t»! - H — yCA,(  V  <AT). .«!"*  I  <AT) 

J  J  3  A  1  *-J  1  3  1  3 

3  2  for  j>l 

j*l . ,m 

vi+1  *  (v1*«(ATu,+1-v,))+ 

wi+1  a  (w^  -  w(u^  +  w^  -  e))+ 

3.2  LIS0R(A,  b)  Convergence  Theorem  For  each  e  >  0  the  iterates  (u*,v*,w*) 
of  the  LIS0R(A,  b)  algorithm  are  bounded  and  have  an  accumulation  point 
(u(e),  v(e),  w(e)).  For  all  ee  (0,i]  for  some  i  >  0,  the  point  (x,y) 


(3.11) 


x:*  t-(-ATu(c)  +  v(e)),  y:«  7 (u(e)  +  w(e)  -  e) 


Is  Independent  of  e  and  Is  the  unique  solution  of  (3.5)  with  least  2-norm, 
and  x  Is  the  unique  solution  (3.6)  with  least  ||x,  (Ax-bJJ^- 

Proof  That  the  Iterates  (u^.v^w1)  have  an  accumulation  point  which  solves 
(3.8)  for  all  c  >  0  follows  from  Theorem  2.2  of  [10].  That  for  ec  (0,e] 
for  some  e  >  0,  (x,y)  defined  by  (3.11)  Is  the  unique  solution  of  (3.5) 
with  least  2-norm  follows  from  the  duality  equivalence  of  (3.7)  and  (3.8) 
and  from  Theorem  2.1.  Since  problems  (3.5)  and  (3.6)  are  equivalent  and 
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y  *  (Ax -  b)+  for  a  solution  of  (3.5),  It  follows  that  x  Is  the  unique 
solution  of  (3.6)  with  least  value  of  ||x,  (Ax  -  b)+l| 2 •  □ 

We  note  here  Eremin's  algorithm  [5]  which  Is  one  of  the  few  Iterative 

algorithms  capable  of  handling  Inconsistent  Inequalities.  Eremin  gives  no 

computational  experience  and  the  presence  In  his  algorithm  of  a  positive 

00 

stepsize  A.  satisfying  A.  ■*  0  and  F  X,  «  “  may  cause  slow  convergence. 
1  1  1-1  1 

An  Interesting  application  of  the  above  method  Is  to  the  problem  of 
Image  reconstruction  techniques  [7,8]  where  the  fundamental  problem  Is  to 
solve  the  system 

(3.12)  Bx  ■  d  x  >  0 


where  typically  the  m*n  matrix  B  may  be  of  order  28000  x  6000  with  less 
than  IX  of  nonzero  elements  [8].  Iterative  methods  are  well  suited  for  such 
large  sparse  problems.  Unfortunately  such  methods  often  require  assumptions 
that  are  rarely  verifiable.  Typically  such  methods  assume  a  priori  that 
the  system  (3.12)  has  a  solution  [7,8],  In  contrast  our  proposed  LISOR 
method  requires  no  assumptions  whatsoever  when  applied  to  the  equivalent 
problem 

(3.13)  Bx  <  d,  -Bx  <  -d,  x  >  0 


In  particular  LISOR 

of  min  ||Bx-d||, 
xeRJ 


with  least 


will  lead  to  the  unique  solution 
|{x,  Bx  -  d{|2. 


x 
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4.  Normal  solutions  of  possibly  unsolvable  linear  programs 

We  consider  here  again  the  linear  program  (2.1)  but  make  no  assump¬ 
tions  whatsoever  regarding  Its  feasibility  or  solvability.  The  Idea  here 
Is  to  apply  the  LISOR  Algorithm  3.1  to  the  equivalent  linear  complemen¬ 
tarity  problem  for  (2.1)  [4] 


(4.1) 


Mz  +  q  >  0,  qz  <  0,  2  >  0 


where 


(4.2) 


Z  -  £Rn+ffl,  M 


0  A 
•A  0 


and  u  Is  the  dual  variable.  Direct  application  of  the  convergence 


Theorem  3.2  to  the  LISOR 


of  LISOR 


(ft)  •  0) 


algorithm  gives  that  the  iterates 


are  bounded  for  all  e  >  0  and  that  for  all 


ee  (0,e]  for  some  e  >  0  they  lead  to  a  z  which  Is  independent  of  e 
and  such  that  z  Is  the  unique  solution  of 


(4.3) 


min  ||  (-Mz -  q,  qz)+|| 

_n+m  I 


with  least  ||z,  (-Mz-q,  qz )+ (| g.  Hence  z  «  M,  is  an  exact  solution 
of  the  primal-dual  pair  with  least  2-norm  if  the  linear  program  (2.1)  Is 
solvable.  Else,  it  is  the  unique  solution  of  (4.3)  with  least  2-norm  for 
the  vector  In  R2(n+m)+1  composed  of  the  primal-dual  variable  z,  the 
primal-dual  infeasibility  (-Mz-q)+,  and  the  primal-dual  objective 
function  inequality  (qz)+. 


5.  Computational  results 


Computational  experiments  have  been  carried  so  far  on  the  LPSOR 
Algorithm  2.3  only.  Results  on  medium-sized  problems  were  given  In  [11]. 

We  give  below  new  computational  results  for  randomly  generated  large  sparse 
problems  carried  on  the  VAX  11/780  with  double  accuracy  floating  point  addi¬ 
tion  time  of  4.6  us  and  multiplication  time  of  6.0  us.  Comparisons  were  made 
with  Marston's  XM P  revised  simplex  linear  programing  code  [14].  The  results 
shown  In  Table  1  are  all  for  a  matrix  A  with  essentially  a  "trldlagonal" 
structure  and  fully  dense  last  column  and  row.  The  XMP  accuracy  was  to 
within  12-figure  accuracy  of  the  current  objective  function  when  It  managed 
to  obtain  a  solution.  The  accuracy  of  the  LPSOR  was  measured  by  the  <»»norm 
of  the  primal  Infeaslblllty  of  the  numerical  solution  and  the  relative 
deviation  of  the  computed  maximum  value  from  the  true  maximum.  The  table 
Indicates  that  for  the  accuracy  obtained,  the  LPSOR  method  becomes  competi¬ 
tive  with  the  simplex  method  as  the  problem  size  gets  larger  and  that  for 
very  large  problems,  SOR  methods  may  be  the  only  viable  methods  of  solution. 

Table  1 

Comparison  of  the  Revised  Simplex  Code  XMP  and  LPSOR  for  Solving  2.1 
m  ■  no.  of  Ineouallty  constraints,  n  ■  no.  of  nonneeatlve  variables 


XMP  _  _ LPSOR 


m 

£ 

Iteration  No. 

Hr:M1n:Sec 

Iteration  No. 

Hr:N1n:Sec 

Relative  Accur 

100 

200 

123 

0:00:11 

180 

0:00:17 

10'6 

500 

1,000 

746 

0:03:12 

520 

0:05:11 

10'9 

1,000 

1,000 

2,309 

0:42:02 

1,640 

0:26:12 

10’4 

2,500 

10,000 

Could  not  solve*) 

480 

0:37:25 

10"4 

5,000 

20,000 

Could  not  solve  *>) 

660 

1:17:53 

10*4 

a)program  was  killed  after  more  than  3  hours  of  CPU  time. 

^Program  used  virtual  memory  space  much  larger  than  physical  memory,  so  It  ran 
Inefficiently  and  had  to  be  killed  within  10  minutes  of  CPU  time  which 
corresponded  to  over  8  hours  of  real  elapsed  time. 
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